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ABSTRACT 

During the past 20 years, international and bilateral 
assistance and funding agencies have become increasingly involved in 
evaluation activities in adult and nonformal education, especially m 
developing countries. The adult education field pioneered many 
approaches now advocated by specialists in prograjh rnd pro3ect 
evaluation. Beginning in the 1920s and 1930s, Unites? States 
agricultural extension specialists did extensive studies on farmer 
extensioir-pfac^i-ce^r^rri-fhe^l^ and 1960s, international effc.rts 
began to build on some of the earlier agricultural extension 
concepts. Needs assessments, evaluation of adult education booklets, 
and evaluation of UNESCO adult education, literacy, and reading 
materials were all conducted. The UNESCO Experimental World Literacy 
Program conducted 11 projects in the late 1960s to mid-1970s that had 
built-in evaluation systems. During the 1970s and early 1980s, the 
governing bodies of most international organizations stipulated more 
evaluative efforts to provide data on the impact of various 
organizational activities. Such evaluation efforts sensitized pro3ect 
and program personnel in international agencies and m national 
governments to the need for participatory evaluation specifically to 
improve projects and programs. (A list of references is appended.; 
(YLB) 
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In education, until recent years, the term "evaluation"., 
has "been exclusirely identified with the measurement of student 
achievement. In the sixties and seventies, however, the notion 
of project and program evaluation has gained momentum, largely 
because of the increasing pressures tram legislative >9dies of • 
nations and governing hodies of national and multi-national 
organizations to accotsnt for funds invested in education. 

By the ei^ties, literature on project and program 
evaluation was vast. Scholars in every social science discipline 
from economics to anthropology* sociology and psychology and in 
professions from education to pubHc administration to social 
communications have been involved in attempts at conceptualizing 
vhat evaluation is ahd should be all about. • If there is any con- 
sensus among practitioners it is that evaluation can mean many 
things to many people. 

!Ehe most general (and perhaps useful.) description of 
evaluation is that it is the coUection of information useful 
in decision making, but there are those that also so define 
educational planning^O^ia^Ldii^' 197U, 1979). This would suggest 
. a close symbiosis between evaluation and planning. Evaluative 
activities, in such a contaxt, involve collecting information 
useful in project and program design ("needs assessment" and 
"institution building" analyses); useful in making mid-course 
corrections when the project is xaiderway {'formative evaluatioii'^, 
and useful at the end of the project or at certain major periods 



in a prograa vbiux cuaiulative information of summary nature 

vbuia -be useful to policymakers and administrators ("sumiiiative ; . 

eraluation"); 

Other distinctions in rarious definitions suggest a differ- 
ence tetveen "nom-referenced" evaluation vhereljy a program is 
compared to standards previously set and "criterion-referenced" 
vhereljy a project is examined on the hasis of vhether or not it 
is reaching ohjecrtives it has set for itself (Werdelin 1911) - 
More recently, evaluation specialists are suggesting "qualitative" 
or "naturalistic" evaluation approaches, as opposed to the highly 
structured, quantitative designs -advocated .tir.T'armbeT.-.ot 
economists and educational researchers grounded.in econometric •= 
and controlled experimental studies (Guha 198I, Patton 1975. Kinsey 
1981, Clark and McCaffery 1979). The Valit'ti^e" »PProacb 
generally stresses participation of all those involved in pro- 
ject and program activity and is thus often called "participatory 
evaluation." «.This section vill trace the involvement ofinter- -- 
national and hilater«a assistance and funding agencies and groups 
in evaluation activities in adult and non-formal education and -' 
will concentrate on activities in developing .countries. Ho 
- attempt will he made to catalogue evaluation efforts, hut rather 
to identify trends «id sugg'jst selected referaices. Purely 
domestic evaluation activities, such as those funded in the 
United States hy the Hational Institute of Education and other 
groups, are not included a. these are veil covered rn other 
standard references such as the -Encyclopedia of Blucational 



Besearch of the ^erican Educational Pesearch Association. 
1. Early Evaluation Efforts in Adult Education 

A case can he inade that' the adult education field, 
broadly defined, has pioneered many of the approaches nov 
advocated hy specialists in program and project evaluation. 

In the twenties and thirties , agricultural extension , 
specialists in the, United States did extensive studies on the 
effect of various extension practices on farmers. Generally 
speaking, such studies indicated that famers are most likely 
to change practices vhen they see a demonstration of a tech- 
nique on a farm similar to their ovn. Publications, radio' pro-^- 
grazBS, and face-to-face ' visits of extension -workers reinf ofce • 

efforts to change farming practice, but these various techniques 

■> # > 
-often 

in themselves Jessed the mark because the change agent lacked an 
understanding of what the farmer alrea^ knows , thinks , feels , 
and does. Agricxdtural education specialists early practiced 
what would now be called ''formative' evaluation" ' when". they triedr.' 
out extension publications in order to see what farmers could 
read with ease, what words farmers know and xxse and which ones 
are used only by agricultural specialists, etc. 

In the fifties and sixties, a number of international 
efforts began to biuld on some of the earlier agricultural 
extension concepts. JUNESCO's Regional Center for Fundamental 
Education in Latin America (CBEFAL), established in Fatzcuaro, 
Mexico, built into its training program various e:q>eriences 
involving the collection of information on the rural population 
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to be serred by adiiLt education activities • Such approaches 
later vould he called '•needs assessiont" and vould he considered 
iarportant as an element of '•formative evaluation*" 

Also in the fifties, the Organization of American States 
(OAS), tlurough its secretariat, the Pan American Ifeion, estab- 
lished the Latin American Fundamental Education Press, a. program 
involving the preparation and try-out of a. series of adult educa 
tion hooklcts designed for adults of limited reading ability in 
Latin America. Diis series included booklets in health, agri- 
culture, civics, recreation, and other areas considered at the 
time important in Latin Amer^.can development* The series was 
pretested in a number of Latin Ajiericah "countries by surveying 
reading interests in village and urban areas; vorking vith a 
saa^e of adults in various socio-economic settings to see what 
they could read, understand and remember of the booklets being 
prepared; and evaluatin g the effect of varl oos J ormats (e.g., 
illustrations and captions with text) on interest and compre- 
hension- (Spaulding and Nannetti in Richards 1959) » 

During the same period, the Inter-American Institute for 
Agricultural Sciences, headquartered at Turrialba, Costa Rica, 
embarked on a series of activities, in coop<^ratipn with-the OAS, 
involving research and training in agritSLtural comniunications. 
Again, the emphasis was on "needs assessment" (finding out about 
the conditions *.in the communities to be served), on "formative 
evaluation" (pretesting and trying out extension approaches and 
materials), and on "suomative /evaluation" (doing follow-up 
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•studies to ascertain vhat had happened as a resiat of the pro- 
gram). These early efforts led to the establishment of a formal . 
departzoent of agricultiiral communications at La Molina jAgri- 
ctaturai Tfaiversity in Perii, vhich has taken regional leadership 
in training and research in this area. 

UHESCO work in adult education, literacy, and reading 
materials for the nev literate during the fifties and sixties 
stressed Tarious. kinds of CTaluative approaches, often described 
then as ''topic testing/ Vetesting,**, «po«ttesting," "study of 
the. environment,'' etc. (Spaulding 1958). In 1957, UHESCO held a 
regional seminar on- reading materials for new literates- in- Ban-'V- 
goon, Burma, which entailed the , collection, of ^information on village, 
needs, writing and pretesting of reading materials, and evalua- 
tion of the impac^Tof such materials. In 1956, a similar regional 
meeting was held in Murree, Pakistan, which outlined evaluation 

.and research needs in the area of reading materials and literacy 
programs (Richards 1959). -UHESCO, the ForA Poun*.tion and other - 
international groiipa assisted diiring the fifties a number of 
national programs involved in the creation and evaluation of 
adult education literature. Among these vere'the East African 
Literature Bureau, the Burma Translation Society, the Bureau of 
Ghanaian Languages, and the Marbial Valley Project in Haiti. 
The East African Literature Bureau, headquartered in 'Jairobi, 
vas the host, in 1962, of a UNESCO-sponsored regional vorkshop 
on the preparation of reading materials for nev literates which 
included authors, editors, publishers, and artists from both 
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Baglish- and Itench-spealdng African couatries. Participants 
foraed team and worked in tillage areas to identity lieeds as : 
f villagers and community development eacpefts. They then 
designed reading materials which were tried out in villages, . 
revised, and duplicated. 

2». The UMESCO ibcperimental World literacy Program. 

Jn the mid-sixties, UHESCO proposed massive s\Q)port for an 
experimental world literacy program, designed to combine social ^ 
and economic skills training with traditional comnrunication 
skills training. The Italted Hations Develoiment Program (the 
I usvial source of support for such projects vithin the UH system) 

I was approached to help finance the effort i hut the UHDP was dia- • 

■ tinctly cooi to- the idea. Although economists had hegun to treat 
formal education as investment rather than consumption, develop- 
ment thinking at that time had not made a strong case for invest- 
ment in literacy and adult education. UBDP -finally agreed, how- 
ever, to finance several pilot, experimentiO.- projects, on the - 
condition that UHESCO would huild into each project an elahorate 
evaluative system designed to show how each project affected the 
behavior of individuals and comnwnities involved in the functional 
" literacy programs (Spaulding I966). 

Ultimately, 11 experimental projects were funded under the 
experimental effort, most of them hecoming operational in the 
late sixties and terminating hy the mid-seventies. Despite the 
efforts to design compcrahle evaluative procedures in each pro- 
ject beforehand, the local context of each project was such that 
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fev comparable data vere collected. Political problems plagued 
the effort in that most governments vished a national literacy 
effort rather than a pilot experimental project. Some projects- 
separated eraluation trom program development , *:Vuid* evalua-* 
tor (usually & social scientist) kept his or her distance from 
the operational staffs vhile others considered evaluation to he 
more of a formative variety,, with the evaluator collecting data 
to he shared vith program developers in order to help improve 
the project as it progj^eased. ^ 

In the mid--seventi es 9 UHESCO established at headquarters in 
Paris an ^'evaluation unit** idiich labored vithout. access for some.-* 
months in an attempt -to integrate- incomparable-data-of*the -11 
literacy projects into some kind of comparable reporting design. 

Finally 9 two educators vere invited to examine each case, to 

I 

STommarize the findings of each, and to report on trends that vere 
evident betveen and among cases. - The result vas a Joint UBDP/ 
UHESCO report vhich vai generally vell-received by1he scholarly 
community, but vas somevhat controversial vithin UNESCO because 
of its unbiased reporting of the problems as veil as the suc- 
cesses of the experimental program (Spaulding and Gillette 19T6)- 
The report looked' at the political and policy problems that' 
plagued the various projects; the administrative and organiza- 
tional problems; 'the staffing problems; the cost of the projects; 
the. nat\are and effectiveness of the teaching materials and methods 
used; and the apparent effect on people and communities. 
Essentially, the report highlights the complexity of any attempt 
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to xuake aiajor changes in an adult population through any one 
adult education or liteiracy effort; the- need to adjust -any such •. 
program. to local conditions; the need to get policy-leTel support 
at the highest levels for success of any prograa; and the need 
to integrate- such efforts into hroader development plans. In 
terms of changed hehavior on the part of participants, the, 
adults in the various projects vould adopt nev practices only 
if they did not take much tiae, effort, or resources to 
acconrplish. 

3. Current Evaluation Activities 

During- the seventies, and early eighties, the governing- hodi'es 
of Bost-intemational -organizations- called for nor evaluative:. i=-.- 
efforts in order to provide data -which shows the impact of the 
wious activities of the organizations. Both UHDP and UNESCO 
established offices of evaluation which, in turn, undertook to 
Jointly evaluate a number of educational projects. TJKESCO pub- 
lished guidelines on project evaluation to help short-term con--"' 
sultant teams brought in to do mid-term project evaluations 
(UHESCp 1979) and a manual on how to do structured evaluations 
of literacy programs (Couvert 1979). The United States Agency 
for International Development (USAID) established, in October 
1979, a major division for project evaluation vbich has pro- 
duced a remarkable series of reports on various USAID-assisted 
projects in rural education, rural development and related 
fields-, most with adult education components (i.e., Giovanni, . 
Armstrong and Jansen 1981). 
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USAID has also supported a variety of evaluation efforts 
through contracts yith uxiiversities and consulting agencies. / 
The Stanfcard Iftdversity Institute for Conmiunications Research 
and^saeay for Educations! Developoent undertook a seriea of, 
evaluative studies of the out-of-school education efforts in the 
Ivory Coast, a program linked to a major television-hased pri- 
mary school reform- in that country (Grant and others n*dO. 
USAID has also funded the Center for International Education at . 
the Tfaivcrsity of Itossachusetts lAich.has developed collaborative 
non-formsl education programs in a number of countries, most vith 
evaluative approaches built into the programs • (BLnsey 1978, apSl).* 
World Education, Inc., -in Hew Tork-CityV-has pioneeredTdnr-needi.^ 
assessment and participatory evaluation approaches in non-formal 
education in a number of countries and has similarly received 
USAID support (ti'. , Farmer and Papagiannis 1975). USAID, for a 
number of years, -supported work at Michigan State University 
which included a Clearin^ouse on Non-FormaO. Education within • • 
which a number of project reports and evaluative studies have 
heen deposited (Son-Fonnal Information Center 198I). In 1976, 
USAI&heiped establish the Institute for Development Anthro- 
pology at Binghamton, Hew York, which has as its goal the appli- 
cation of social sciences in develojtaent work. It has undertaken 
a number of evaluative studies for USAID of rural and community 
development efforts (Institute for Development Anthropology 1981 ). 

Major studies fanded by USAID and others include a three- 
year effort by Florida State University to evaluate the extensive 



11 



10 



vork of Accifin Popular Cultural (ACPO) in Coloabia. ACPO, over 
the yearly liaa evolved a radiophonio education effort -which 
includes puhlicationa, voluateer monitors in many villages, and 
a variety of adult education activities. vhich are supported hy 
the communications infrastructure. The complexity of developing 
Md managing such a program and attracting continuing interest 
of participants has heen highlighted in this study (Bemal, 
Masoner, and Masoner 1978, Morgan Muhlmann and Masoner 1980). 

DSAID also initiated in the. early eighties a major demon- a 
stration of the uses of satellit'es to improve communication in 
rural a^-com»unity developnent. 'This effort will, have .a major 
evaluation component. Liheria is the location of the first of 
several country projects, each of which emphasizes the role of 
communication in adult education. This effort is consistent with 
an increasing concern of a number of funding agencies for the 
integration of adult education services . into compr-eheusive com- 
aunity developnent efforts. (Coombs 198O). . 

The World Bank has an office which undertakes evaluations 
of projects funded by the various regional bureaus of the Bank, 
but most of these "audits" are for internal use only and are cot 
- aade public. Recently, however, the Bank began to encourage bor- 
rowing nations to build into each project it funds a continuous 
evaluation system which will provide information useful in . 
ioproving project performance as it progresses as well as cumu- 
lative informtion to help in the ultimate summative evaluation 
of the effort. The first such built-in feedback evaluation 
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aechanism, to cost severail aiillion dollars over a sevea-*yeaf 
period, 'Jns developed in Papua Mew .Guinea in 1980 as part of ,a 
primary education Teform loan (vhich also included a small adult' • 
education component )• An earlier such evaluation effort in a 
Bank-supported project in Pakistan was abandoned. 

Ihe Bank, in^he mid-seventies, - assisted the Saudi Arabian 
Government in a major evaluation of their adult education and 
literacy efforts, with the help of faculty from the International 
and Derelojnent Education Program of the University of Pittsburgh, 
the University of Linkoping in Sweden, Ain Shaims University in* 
Cairo> World Educationr-.Inc. ,^and other groins. This. led ta a^ ^ 
major, revision, of the curriculum and materials -xxsed-in^the Saudii- 
program, but the Saudi Government has never released the various 
evaluative and research reports prepared during the project* 

The Canadian International Development Au fc hwi^y (CIDA) has 
participated in a variety of evaluation efforts j often in coopera- 
tion witti the International Council -for Mult .Education in-T6ronto 
and the International Development Research Centre in Ottawa (Inter 
national Council for Adult Education 1979). The Swedish Inter- 
national Development Authority has similarly funded a variety of 
adult education projects with an evaluative component, including 
one to train adult educators in Portugal undertaken by the Uni- 
versity of Linkop^g (Erasmie, Horbeck 1978). The Overseas 
Development Authority in the United Kingdom has fi;nded a number 
of >such overseas projects, especially through the University of 
Heading, which has a long-itanding interest in agricultural and 



13 



12 

rural dcveloiaaent (Bovers 197T). 

. PinaTiy, a joixxt UNiSCO-UNICEF program of studies has 
issued an extensive series of reports vbich includes evaluative 
material on a variety of non-»foraal projects and programs Tor 
children, youth and vomen. And the UNESCO Regional Office in 
Asia hosts the Asian Program for Educational Innovation and 
Development (APEID} vhich has netvorked a nioaber of institutions 
involved in evaluative studies of educational innovation in the 
region. 

Institutional STaluatioii Policies 

As can he seen from' the selective- reyiev,'ahoveV* most Inter- 
national eveiLuat ion' efforts in non-formal and adult '^sducation.- 
have been motivated hy multi-national and bilateral funding and 
assistance organizations. As yet, however, there is little agree- 
ment among and between agencies and governments as to what evalua- 
tion is all about. ' . 

Some in the-agencies-see evaluation as an exercise^.yhereby 
"experts" are sent for a few days or weeks to look at a project 
in progress to decide whether or not it is meeting its objectives 
and whether or not any changes are needed in the project plan* 
Governing boards of international and funding organizations, in 
turn, see evaluation as a means of providing data to help them 
set priorities as to vhat should be funded in the futiire. 
National govsmments, project administrators and groups involved 
in projects, however, are more attracted to the idea of evaluation 
as a kind of jcanagement information system which can help pro- 
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vide data for ongoing improvement in project management and 
design. 

Ihe picture is further complicated by the internal structtires 
of international and funding organizations • Within the opera- 
tional arms of many organizations, in-house evalnators '(some- 
times called "inspectors general** or ^auditors") are often viewed 
as meddlers* If the evaluation unit of the organization is not 
placed at a high enougjx level in the organization, and given suf- 
ficient resources, -.autonomy and authority to do its vork, its vork 
will be limited and its information will be little used in policy 
ff»;Vin g and operational decisionmaking.f -■ 

Perhaps wore importantly., evaluation efforts, to.be effective 
in improving program and project performance, must be built into 
program and project activities with the idea of helping answer 
question.'! which participants have. External evaluations done by- 
evaluation specialists may serve .a function, but until evaluation 1. 
is a state of mind shared by all participants in- a program, 
evaluation results will be- less than fully effective. For this 
reason, a major function of any in-house evaluation unit should 
be to help operational units within the organization in the 
design of their own self-e-valuation efforts. 
5. The Future 

The past two decades have sensitized project and program 
personnel both in international agencies and in national govern- 
ments to the need for evaluative data on pi«oject and program 
activities. With .increasing interest world-wide in non-formal 
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and adwXt -education, there vill "be increasiag interest in 
evaluation efforts designed to provide information foi the 
constant inprorenent of projects abd programs. ^Evaluation 
models viU increasingly stress participation of the target 
audiences of non-formal and adult education efforts. Such 
participatory evaluation in. itself vill be a poverful ad.ult : 
education nediun, consistent with current priorities on the 
developaent of community-'based non-formal education efforts 
reflecting the needs and interests of local cooniunities . 
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